PCNN: Projection Convolutional Neural Networks

49

Conv 3×3

Conv 3×3

+

l^x

1

l^x 

Conv 1×1

Conv 1×1

+

Conv 3×3

l^x

1

l^x 

MCconv 3×3

MCconv 3×3

+

l^x

1

l^x 

(a) Wide-Resnet

basic block

(b) Wide-Resnet

bottleneck

(c) MCN basic

FIGURE 3.9

Residual blocks. (a) and (b) are for Wide-ResNets. (c) A basic block for MCNs.

M-Filters can capture the hierarchy and diverse information, which thus results in a high

performance based on compressed models. Figure 3.11 show the curves of the elements in

M-Filter 1 (M ^′

1^{), M-Filter 2 (M ′}

2^{), M-Filter 3 (M ′}

3^{), and M-Filter 4 (M ′}

4^{) (in Fig. 3.2(a)}

and Eq. 3.12) on the CIFAR experiment. The values of nine elements in each M-Filter are

learned similarly to their averages (dotted lines). This validates that the special MCNs-1

with a single average element in each M ^′

j ^{matrix is reasonable and compact without perfor-}

mance loss.

3.5

PCNN: Projection Convolutional Neural Networks

Modulated convolutional networks (MCNs) are presented in [237] to binarize kernels,

achieving better results than the baselines. However, in the inference step, MCNs require

FIGURE 3.10

Training and testing curves.